Octupolar order in the multiple spin exchange model on a triangular lattice 
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We show how a gapless spin liquid with hidden octupolar order arises in applied magnetic field, in 
a model applicable to thin films of '^He with competing ferromagnetic and antiferromagnetic (cycfic) 
exchange interactions. Evidence is also presented for nematic — i.e. quadrupolar — correlations 
bordering on ferromagnetism in the absence of magnetic field. 
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The triangular lattice occupies a special place in the 
history of quantum magnetism. Although it is the sim- 
plest lattice to exhibit geometrical frustration, this still 
has highly nontrivial consequences — notably, that the 
Ising antiferromagnet (AF) on a triangular lattice re- 
mains disordered even in the limit of zero temperature. 
This fact led Anderson to suggest that the ground state of 
the spin- 1/2 Heisenberg AF on a triangular lattice would 
be a gapless spin-liquid state with "resonating valence 
bond" (RVB) character While this conjecture has 
since been disproved [H, the idea of the RVB state con- 
tinues to exert a strong hold on the popular imagination, 
and has motivated a search for spin-liquid states in a wide 
variety of two-dimensional spin models 

Lately, attention has been refocused on the trian- 
gular lattice by the discovery of a number of exper- 
imental realizations of a quasi-2D spin-1/2 magnets 
with triangular coordination. These include the organic 
(BEDTTTF)2Cu2(CN)3 the transition metal chlo- 
ride CS2CUCI4 0, and sohd films of '^He absorbed on 
graphite Q. Of these, '^He films offer the most perfect 
realization of a spin-1/2 triangular lattice, but with an 
interesting twist — the nearest-neighbor interaction is 
ferromagnetic (FM) and competes with an AF 4-spin 
cyclic exchange [7] . As such, '^He on graphite is one of a 
number of new quasi-two dimensional systems to exhibit 
"frustrated ferromagnetism" [^, 0| . However it is unique 
in both its purity, and in the possibility of tuning the ra- 
tio of the competing interactions continuously simply by 
varying the density of "^He atoms. An additional strong 
motivation for understanding this system comes from the 
fact that, for a range of densities bordering on ferromag- 
netism, the magnet ic g round state of ^He films is a fully 
gapless spin-liquid 



FIG. 1: (Color online). (Left) Schematic phase diagram for 
the spin-1/2 J-K multiple spin exchange model on a triangu- 
lar lattice in applied magnetic field, in the parameter range 
where J < is FM and /(T > is AF. A "triatic" phase with 
magnetic octupolar order appears in the shaded region be- 
tween fully polarized and canted AF (CAF) phases. (Right) 
Schematic spin structure of this triatic order, which is invari- 
ant under spin rotations of 27r/3 about the z axis. 
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Recently, we proposed that the competition between 
FM and AF interactions in frustrated spin systems could 
lead to a new kind of gapless spin-liquid state with hidden 
multipolar order bordering on the FM phase [H [H Q 
(for related, earlier work see In this Letter we ex- 

tend these ideas to a triangular lattice spin-1/2 model 



where P2 and P4 (cyclically) permute spins on nearest 
neighbor bonds (ij) and diamond plaquettes (ijkl), re- 
spectively. We include a coupling to magnetic field h, 
and consider exclusively FM J < and AF K > 0. 

Our main finding is that a dynamical process unique 
to 4-spin exchange on the triangular lattice leads to the 
formation of three magnon bound states which — un- 
der applied magnetic field — condense, giving rise to 
a new, partially-polarized quantum phase in which ro- 
tational symmetry is spontaneously broken, but with- 
out any long-range spin order perpendicular to the field 
(Fig. [T|) . We identify the order parameter for this novel 
phase as a rank three tensor with octupolar character, 
and confirm its existence through the exact diagonaliza- 
tion of finite-size clusters. 

To better understand the origins of this new quan- 
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FIG. 2: (Color online). The four-spin cyclic exchange op- 
erator can be used to propagate three flipped spins (•) in 
a polarized FM background by tunneling between compact 
triangular and straight line configurations. 



turn phase, let us first reconsider the classical S oo 
limit of Eq. For K/\J\ < 1/4, the ground state 

is a saturated FM. For 1/4 < K/\J\ < 1 it is highly 
degenerate [l^. Freely propagating domain walls de- 
stroy long range spin order, but nematic (i.e. quadrupo- 
lar) order persists [l3|. This pathology is also evi- 
dent in the spin excitations for the quantum spin-1/2 
case. Within the FM phase, the one-magnon dispersion 
w(q) = -{J + AK) [3-cos{qx)-2cos{qx)cos{\/3qy/2)] 
is well defined. However exactly at the classical critical 
point J = ^AK it vanishes identically. 

Nondispersing — i.e. localized — magnons are also 
found at the border of a FM phase in the equivalent 
multiple spin exchange model on the square lattice. In 
this case, pairs of flipped spins can propagate coherently 
under the action of AF spin exchange, even where a 
single magnon is localized. Magnons therefore form ki- 
netic energy driven bound states, which condense and 
give rise to a quadrupolar "bond-nematic" order [14 1. 
The energy of two flipped spins in a FM background 
can also be calculated exactly for the multiple spin ex- 
change model on a triangular lattice. We find that two- 
magnon bounds states with total spin S = iV/2 — 2 and 
momenta q = {(0, 27r/V3), (tt, 7r/-\/3), (— tt, 7r/-\/3)} have 
a lower energy per spin than independent magnons for 
K/\J\ > 0.2349. 

However, the frustrated geometry of the triangular lat- 
tice introduces a new, two-step process by which a group 
of three flipped spins can move together — as shown in 
Fig. [21 This motivates us to try a simple trial wave func- 
tion for a three spin bound state, within the restricted ba- 
sis of states where the three spins form a compact straight 
line or triangle. Solving the Hamiltonian Eq. ([T|) in this 
restricted Hilbert space, we obtain five eigenmodes for 
the three-magnon bound states. The lowest eigenvalue, 




E = -13J-50K-VJ'^ + AJK + 28K^, corresponds to a 
state with momentum q = (0,0), belonging to the trivial 
irrep of the space group, i.e. even under the space ro- 
tations R2tt/3, Rtt and the reflection a. This variational 



FIG. 3: (Color online). Critical fields hd (dotted line), hc2 
(dashed line), and /ics (solid line), where one-, two-, and three- 
magnon instabilities destabilize the fully polarized state in 
the J-K model with FM J < 0. The lines of hd and hc2 are 
given from exact solutions. The variationally estimated line 
hcs gives an exact lower bound for three-magnon instability 
line. 



estimate of the three-magnon bound state is lower in en- 
ergy than any single magnon or two-magnon state for a 
range of parameters 0.2347 < K/\J\ < 0.2652. 

To put these calculations in a more physical context, 
let us consider the case in which a magnetic field h is used 
to fully polarize the system. This field is then reduced 
until a critical value he is reached, at which point the 
magnetization starts to change. We can calculate critical 
fields hci and hc2 at which one- and two-magnons start 
to Bose-condense. We can also estimate — using the trial 
wave function discussed above — the field hcs at which 
the system becomes unstable against the condensation of 
three-magnon bound states. In the absence of any other 
phase transition, the greatest of the three fields hd , hc2 
and hc3 will determine the first instability of the system, 
and the nature of magnetic order near to saturation. 

Performing this analysis, we find that the first instabil- 
ity for K/\J\ > 0.27 occurs in the one-magnon channel, 
and is against a three-sublattice canted AF, as previously 
found in mean field analysis [l6l | . However in the param- 
eter range 0.2347 < K/\J\ < 0.27, h^s > K2 » hd — 
see Fig. [3] Since the estimate of ft.c3 is variational (while 
the values of hc2 and hd are exact), it provides a strict 
lower bound on he- 

Exact diagonalization of finite clusters of up to iV = 36 
spins confirms that the transition out of the saturated 
state is controlled by three-magnon bound states for 
0.24 < K/\J\ < 0.28. Results for a 36-spin cluster with 
K/\J\ = 0.25 are shown in Fig. [H Jumps of |Am| — 3 
are clearly visible in the magnetization process (inset) 
for fields close to saturation {Hg = /ics). The strong 
AF coupling K ensure that interactions between three- 
magnon bound states are repulsive, and excitations con- 
taining more than three magnons therefore play no part 
in the transition. At finite Hg, the transition is of second 
order, and can be understood as a Bose-Einstein conden- 
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FIG. 4: Energy spectrum of the J-K model for A'^ = 36 spin 
cluster with J = — 4 and K = 1, showing the oscillation in 
energies of the lowest lying states in spin sectors in the period 
three for large S. The inset shows the magnetization process 
m/ms. Jumps of |Am| = 3, corresponding to octupolar order, 
are clearly visible at large H. (Color online). 



sation of three-magnon bound states. For Hs — > 0, this 
gives way to a first order transition. 

These jumps can be traced to a set of states which lie at 
the bottom of spin sectors S = 3n, belonging to the triv- 
ial irrep of the space group. Each jump of Am = — 3 cor- 
responds to adding one more three-magnon bound state 
to the system. The convex locus of these states as a func- 
tion of 5' imphes that the transition at hcs is of second 
order. Numerical and analytic results close to saturation 
are therefore in perfect agreement. The condensation of 
three-magnon bound states will lead to a new form of 
quantum order. But what kind of order is it ? 

We consider first the situation in applied magnetic 
field, where full SU{2) spin symmetry is broken down to 
U{1) rotations about the z-axis. In this case the order pa- 



rameter is determined by the operator O, 
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which creates a three-magnon bound state in a polarized 
background. Both Tie and Im are linear combi- 
nations of fully symmetrized rank-3 tensor operators 



E 

a,fc,ceP(Q,/3,7) 



Qa QO QC 



(2) 



and the resulting order parameter corresponds to a mag- 
netic octupole on a triangular plaquette {ijk}. 

As expected, both the real and the imaginary parts of 
the order parameter transform with period 27r/3 
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where Rg denotes a rotation in spin-space through angle 
9 about z axis. The three-magnon condensate breaks 
the remaining U{1) symmetry down to the symmetries 
of a triangle in the plane perpendicular to the applied 
field (Fig. [1]). We therefore dub the resulting octupolar 
order "triatic" . We note that triatic order of this type 
does not require any breaking of translational symmetry 
— the condensed three-magnon bound states can move, 
as long as they remain phase coherent. Triatic order of 
various types has previously been discussed in the context 
of the Heisenberg AF on a kagome lattice [l7i |. 

Period-3 structure in the energy spectrum persists 
down to low spin in all of the clusters studied. How- 
ever, new features appear in the structure of the spec- 
trum for S* < 6 (see Fig. S]). These features are sensitive 
to cluster geometry, which suggests that different types 
of order compete with octupolar order in the absence of 
magnetic field. While this makes interpretation of the 
spectra more difficult, it is nonetheless possible to place 
strong constraints on possible ground states. 

First, it is possible to conclude that a new ground state 
exists between the FM phase and the short-ranged RVB 
spin liquid phase with finite spin-ga p A ^ |J|, which 



181 . For 0-235 



< 



is known to occur for K/\J\ « 0.5 
K/\J\ < 0.28 the ground state and low-lying excitations 
have different symmetries from the RVB phase |20| . Fur- 
thermore, in this parameter range, the rapid decay of 
excitation energies with system size suggests a gapless 
ground state for /i = 0, and thus a breaking of SU{2) 
symmetry. 

Dipolar (Neel), quadrupolar (nematic) and octupolar 
(e.g. triatic) magnetic order all break SU{2) symmetry, 
and each of these forms of order is associated with a spe- 
cific "Anderson tower" of low-lying states . The An- 
derson tower contains states in all spin sectors S for Neel 
order; only states with even S (period-2) for quadrupo- 
lar order, and only states in sectors with S a multiple of 
three for pure octupolar order (period-3). 

In view of the spectra, Neel order seems improbable. 
The number of low-lying states below the continuum in 
each spin sector S is not compatible with what would 
be expected for dipolar order. Furthermore, a period-2 
structure where the lowest lying states with odd 5* are 
raised in energy relative to states with even S, is seen 
for 5 < 6 in those = 12 and A^ = 36 spin clusters 
which possess the full symmetry of the lattice, and in the 
A^ = 24 cluster with the lowest ground-state energy. This 
period~2 structure suggests quadrupolar order is present 
in the ground state. The period-3 structure, present at 
larger S, is still clearly visible for small S in some A^ = 24 
clusters, but is hard to distinguish in the lowest-energy 
clusters. Moreover, the ground states for the A^ = 12, 36 
clusters are not in the trivial irrep, which indicate that 
the symmetry of the ground state differs from the pure 
octupolar order observed in magnetic field. 

In fact, the symmetry and momenta of the low lying 
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FIG. 5: Bond nematic correlation in the ground state of the 
J-K model for A'^ = 36 spin cluster with J = —4, K = 1. 
(See Ref. [TJ] for the definition.) Correlations are measured 
relative to the reference bond 1-7. Positive (negative) corre- 
lations are drawn as full (dashed) lines. The thickness of the 
lines is a measure of the strength of the correlation. 



even-S" states, and the corresponding quadrupolar corre- 
lation function (shown in Fig. [5]), suggest the superpo- 
sition of two distinct types of quadrupolar order. One 
corresponds to the condensation of s-wave magnon pairs 
with momenta at the K points (e.g. q = (0, 27r/'\/3)) of 
the Brillouin zone — precisely the two-magnon instabil- 
ity of the saturated state predicted to occur at hc2- The 
other corresponds to (ij:2_j,2+id£cj,~wave magnon pairs 
with momentum q = (0, 0). 

We therefore conclude that the model has weak 
quadrupolar order in its ground state which extends from 
K/\J\ « 0.23 to K/\J\ « 0.28, where it undergoes a 



phase transition to a gapped RVB phase (2l[. However 
in this parameter range, this gapless nematic phase lies 
close to the boundary with other competing phases. 

We note that octupolar and quadrupolar orders are 
not exclusive: octupolar order is known to be accom- 
panied by quadrupolar order in the ground state of 
spin 5 = 3/2 models with polynomial interactions [3]. 
This raises the question of the possibility of some sim- 
ilar coexistence in the present S' = 1/2 model. Short- 
range FM correlations on individual triangular plaque- 
ttes (((Si + S2 -I- 83)^) = 2.42 for N = 36, K/\J\ = 0.25) 
persist within the ground state, reflecting the tendency to 
form three-magnon bound states. However, the present 
cluster sizes are too small to answer this question. 

The gapless spin liquid state observed in '^He films oc- 
curs in the vicinity of the FM phase. It seems reasonable 
that a nematic phase (or octupolar phase) should inter- 
polate between FM and RVB phases in the multiple spin 



exchange model on the triangular lattice, and that this 
state is therefore a strong contender to explain the spin- 
liquid seen in experiment. 
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We also note that the spectra showed no signature of the 
half-magnetization plateau known to occur in the RVB 
phase [lq |. 

It is also interesting to note that the gapful RVB 
wave function contains states with crossed singlet/triplet 
structure which is also seen in two-spin bound states near 
to saturation [13j |. 



